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Introduction 

Many feminist scholars have criticized an im- 
plicit axiom 7 in research and evaluation "that the 
choice of a problem is determined by method instead of 
a method being determined by the problem" (Daly, 1973). 
In particular, when this unstated but traditional em- 
phasis on the method, rather than on the program or sub- 
jects being studied, has been applied to feminist pro- 
jects,,, the investigations have ended jiip being on women 
and not for women (Duelli-Klein, 1980) . Indeed, the ac- 
cepted, male-dominated view of research and evaluation 
has led Westkott (1979) and Eichler (1980) to criticize 
in traditional investigations the types of questions 
asked; the techniques used; and the conclusions drawn. 
Thus, they have' upged that appropriate methodologies for 
the study of women's projects be developed. 

Aside from evaluative methodologies, criticism has 
also been directed at the obj ect/subj ect split between 
the evaluator and those being assessed. In partici- 
patory democratic settings of most women's projects, 
Duelli-Klein questions whether the detached, neutral 
outside evaluator can be acceptable to those involved 
in the project being studied. The traditional remote- 
ness and lack of involvement of the evaluator may be in 
conflict with the ^high level of commitment of those 
within the project. ^ In addition, the removed, "objective" 
evaluator without understanding and sympathy for the 
types of societal changes that the subjects are trying to 
make may actually carry an unintentional bias into the • 
evaluation. Passivity and neutrality can hinder the de- 
velopment of a rapport between the researcher and those 
being studied. In fact, most feminists-4tave not ex- 
pected an evaluator to approach a new project free of 
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preconceived ideas. Some have introduced the term of 
"conscious subjectivity (Coyner, 1980) which permits 
different perspectives to be presented and acknowledges 
that evaluators also may speak. 

The emphasis on feminist methodologies and "conscious 
subject ivity 11 necessitates that the evaluative mode of 
feminist programs is designed to meet those specific re- 
quirements. This paper examines the \ise of illuminative 
evaluation in a Women's Studies! 1 institute, in which both 
intense commitment and philosophical perspectives a^e 
likely to generate an unusually strong desire among both 
^participants and staff#fQr collaboration in the evaluative 
process itself. This type of assessment was selected be- 
cause of its flexibility and varied evaluative techniques. 

As defined by Parlett and Hamilton (1977), "illumi- 
. native evaluation is not a standard methodological pack- 
age but a general strategy. It aims to be both adaptable 
and eclectic. The choice of research tactics follows hot 
from research doctrine, but from decisions in £ach case as 
to the best available techniques; the problem defines the 
methods, not vice-versa . 11 

Another aspect of 1 illuminative evaluation is that it 
makes no claim bo be value free nor to be capable of total 
objectivity. But it does try to represent diverse be- 
lief s , ideologies , and opinions that may be encountered in 
the course of the study. It also attempts to represent 
differences of thought and perspectives in ways considered 
to| be fair by those holding the views. Still another part 
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duality of the evaluator's role - of someone who needs to 



the illuminative model is the acknowledgement of the 
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become the knowledgeable insider and at the same time 
preserve the independent status of the r outsider . Also, 
illuminative evaluation is clearly interventionistic in 
that it attempts to promote changes in the way people 
view educational processes (Parlett & Dearden, 1977) . 
With its emphasis on change, it has been often recom- 
mended for the assessment of innovative programs 
(Shapiro, Secor, & Btitchart, 1981). Additionally, 
illuminative evaluation's varied techniques can pro- 
vide both program planners and funding ^igerfcies with 
the kinds of quantifiable data they of ten* require . 
« Finally, it, can offer a picture of the often subtle 
but important qualities of the projept which are 
essential to its full understanding. 

Theoretically, illuminative evaluation seemed 
appropriate as an approach for assessing strengths and 
weaknesses, and for assisting the planners in their 
understanding of the dynamics and shape of the First 
National Summer Institute in Women's Studies. In line 
with much feminist * thinking, the illuminative evaluative 
process was focused less on pre-determined method- 
ologies and more on the utilization of techniques ap- 
propriate to participant and staff interactions; par- 
ticipants could be involved in the assessment process; 
no claim of perfect objectivity was made as it was as- 
sumed that the evaluator would function as a participant 
observer, not as a distant, impartial outsider. Thup , 
in theory, illuminative evaluation seemed to have k 
special merits and to be compatible with both staff and 
participants 1 needs for involvement with the evaluative 
process. In practice, this study describes the implemen 
tation of illuminative evaluation and attempts to assess 



the value of this procedure for the special needs of 
pirogram developers, staff and participants, 

Purpose of the Study 

This investigation examines the development of an 
illuminative evaluation of an intensive summer institute 
in Women's Studies. In this proposal, two questions are 
considered: 

(1) To what extent can and should the evaluator 
maintain an independent and objective role 

— ^ within a participatory feminist institute? 

(2) In what ways and why is illuminative evalua- 
tion a suitable methodology for assessing 
feminist projects? 

Description, of the Feminist Institute 

The First National Summer Institute in Women 
Studies was sponsored by the Great Lakes Colleges 
Association (GLCA) and took place on the campus of The 
University of Michigan. The GLCA received a grant from 
the Lilly Endowments to provide substantial support for 
the first two years of the Institute, beyond which it 
was intended to be seXf supporting. 

Basic components of this three week institute in- 
cluded: 1.) a daily theory seminar, meeting in groups 
.of twelve; and 2.) a daily teaching and curriculum 
workshop, meeting in groups of eight. Optional work- 
shops, research and methodology sessions, and special 
interest group activities were offered. Additionally, 
lectures, panels, special performances and films were 
available. Each participant was also pursuing a pro- 
ject in course design* or some other form of Women's 
Studies curriculum development. 



Forty-eight participants, thirteen staff members, 
and two administrators (a director and a local arrange- 
ment person) comprised the institute. Forty-six partici- 
pants were women and two were men. Three of the partici- 
pants were from outside of the U.S.A.. Fifty per cent 
were 31-40 years of age and twenty nine per cent were 
41-50 years old. Almost three-fourths of those involved 
received 100% support from their institution to attend. 
They came from state universities, community colleges, 
women's colleges and private co -educational liberal 
arts colleges. Over thirty-five per cent of the partici- 
pants were assistant professors or lecturers; thirty per 
cent were full professors or associate professors. When 
questionned, seventy per cent of them considered their 
major responsibility to be teaching, although many were 
involved in Women's Studies or other areas of academic 
administration . 

While the participant group lacked racial and/or 
ethnic diversity, two of the thirteen staff members were 
Black women from the U.S. and one was Chilean. Staff 
members and administrators of the Institute were actively 
involved in Women's Studies activities on their own 
campuses . 

In pre-Institute planning sessions, the" staff agreed 
that the major components of the Institute should take in 
to account the following issues: sex discrimination, 
racism, class bias , and the implications of institu- 
tionalized heterosexuality . 

The Institute itself was an intense experience, 
focusing on scholarly, political and personal issues. In 
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designing individual projects, participants were asked to 
consider the impact within their own college or university 
settings. A less explicit /intention was to encourage addi- 
tional work toward institutional and/or social change. A 
range of feminist perspectives were presented in both the 
theory seminars and in the teaching- workshops . Especially 
in the latter, the critical areas of race, class and sex - 
gender were discussed and debated. Throughout the Insti- 
tute, traditional assumptions of the academy were chal- 
lenged. 

Academic colleges and universities which have almost 
totally omitted attention to women's lives and experiences 
in their curricular and administrative policy and proce- 
dures until recently have only painfully and slowly begun / 
to change. The work of challenging the disciplines and 
changing the curriculum has barely begun. Yet working to- 
ward creating an inclusive academy, not focused merely on 
white privileged males, is highly exciting, energizing the 
interest of an increasing number of men as well as women. 
The commitment of participants in such an Institute was 
therefore certain to be intense since feminist theory and 
practice were upper -most in people's minds . Thus, most in- 
dividuals were not only focused on their own participation 
in the Institute, but were concerned with the assessment 
process itself. In this highly critical environment many 
asked either directly or indirectly: who was to evaluate 
this intensive experience ; how was it to be done ; and 
what would happen with the data collected during the 
period ? 

In this intellectually stimulating setting, all / 
assessment processes and procedures derived from the ^ocial 
sciences wer x e challenged and discussed. Evaluative neu- 



trality and objectivity were argued against. Bias was 
thought to be inevitable. Involvement at all stages of 
the assessment process, by participants, staff and ad- 
ministrators, was desired and requested. 

Description of the Evaluation 
Ev aluation Consultant 

From the outset, the director of the Institute en- 
visioned the evaluation of the project to be a collabo- 
rative process. Even in the proposal requesting funds 
for the Institute, she spoke of the assessor, not as an 
evaluator, but as an evaluation consultant. In her own 
mind, a distinction was made, and no outside, impartial 
observer was considered. Instead, a collegiality was 
sought. Although continous collaboration throughout the 
evaluation was never discussed, it was evident to the 
evaluation consultant that some type of collaborative 
venture was intended. 

A model' for illuminative evaluation, consisting 
of a combination of qualitative and quantitative techni- 
ques, was designed. Quantifiable information was ob- 
tained through a needs' assessment form, administered to 
participants and staff, through observations of the for- 
mal and informal sessions and staff meetings, and through 
open-ended questions on the evaluation form. 

The evaluation consultant was on-site for an eight 
day period - four days at the beginning of the Institute 
and four, at the end. She also kept iruclose telephone 
contact with the program director. 

AlthougtiMt was recognized that feminists might 
want to takesffart in the evaluative process, the extent 
of that involvement came as a surprise to both the eval- 



uation consultant and the Institute director. From the 
very outset, program planners, staff and participants 
indicated that they wished to be a part of every aspect 
of the assessment process - from the formulation of 
questions to the staffs' preparation of materials they 
wished to have included in the final report. Often 
their requests for collaboration were granted. 

An example of this type of intensive involvement 



of the final questionnaire 
final days of the Institute, 



pan be seen in the development 
for participants. Toward the 
the evaluation Consultant designed an assessment instru- 
ment. She then showed the director and advisor of the 
project the form, and together significant revisions f 
were made, in both the language and content of the 
questionnaire. Subsequently, the director and advisor 
asked that all staff review the . instrument . Each 
staff member proposed changes. Additionally, two of the 
teaching staff shared the questionnaire with their work- 
shop participants, and further revisions were suggested. 
The evaluation consultant analyzed and synthesized these 
suggestions and modified the original instrument markedly. 
Only one hour prior to its distribution was the question- 
naire completed! ( 

t 

Evaluation Process 

Below are summarized some interactions which oc- 
curred between the evaluation consultant and the program 
planners, staff , and participants during and immediately 
after the Institute. 

1.) Between evaluation consultant and program plan- 
ners - The planners communicated with the evaluator 
throughout the evaluative process; they helped to develop 
the needs' assessment form and refined the questionnaire; 



10 



they assisted in the selection of staff to be inter- 
viewed; they indicated "the importance of staff and 
participant involvement in the assessment process; they 
designed and organized the sessions in which the staff 
would give their final evaluations. 

2. ) Between the evaluation consultant and the staff - 
Selected staff members were observed by the evaltiator in 
their theory and teaching sjeminars ; selected staff were 
interviewed in an unstructured format; selected indi- 
viduals were asked to contribute to the formulation of 
items for the f^inal questionnaire; all personnel assis- 
ted in the refinement of the\ questionnaire before it was 
distributed; all Ltaff met in small groups at the con- 
clusion of the Iit6titute"to evaluate both the structure 
and the process, and to make recommendations for future 
institutes (the staff then met as a whole for reporting 
and synthesizing data frojn the small groups); all staff 
numbers made recommendations for future institutes. 

3. ) Between the evaluation consultant and the partici- 
pants - Selected participants were observed in seminar and 
workshop sessions and interviewed in an unstructured for- 
mat; certain participants asked to be interviewed; some 
participants critiqued the evaluative techniques and asked 
for modifications (some of which were granted, some were 
not) . 

■ The Final Report 
In collaborative setting, the final report of the 
Institute posed two major problems: 1\) For whom was it 
intended? 2 . ) How should it be used? \ 

From the outset of the project, th\e director of the 
Institute wanted evaluative data for three purposes 
which are listed here according to priority. First of all, 
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she requested qualitative and quantitative data to im- 
prove the Institute in its second yearr^ Next, she want- 
ed to provide useful information for the president of her 
organization and for the foundation officer of the Insti- 
tute. Finally, she felt that it was important to docu- / . — 
ment this experience for historical reasons. 

Taking into account the needs expressed by the di- 
rector ior evaluative data, the evaluation consultant 
focused a great deal of attention on information col- 
lected f^r the purposes of strengthening and improving / 
the design of year \t^o of the Institute. She also attemp- I 
ted to synthesize eripugh data to satisfy the requirements ~" 1 

of the president of the director's organization and the J ; 

foundation officer and to provide a written record of The 
First National Women's Studies Institute. 

Discussion of the Results' and Conclusion 

In a highly participatory environment in which the 
focus is on feminist issues, the evaluation consultant 
learned that total neutrality -and objectivity were 
neither expected nor desired by those involved in the 
project. During the Institute, the evaluator had to 
make decisions which resulted in her becoming moi^ of a 
participant and less of an observer, for she was gaining 
knowledge which could help change the dynamics of the 
Institute as it was happening. 

The involved procedure, previously described, of ^ 
designing the final questionnaire was one indication of 
the collaboration which was expected from her. Although 
the process of conferring with Institute personnel and 
participants was time-consuming and taxing for the evalu- 
ative consultant, the questionnaire was much clearer and' 
more inclusive 'because of it. Further, those being 
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assessed felt directly involved in the evaluative pro- 
cedure. The evaluation consultant noted that few com- 
plaints were made concerning ^t he -questionnaire itself 
and some participants specifically expressed positive 
comments about it. - However, it was interesting to note 
that a few individuals whose comments had not been e- 
licited in the early stages of revising the instrument, 
expressed dislike of the questionnaire. Involvement 
seemed to make a difference in reactions for and against 
the instrument. 

From an evaluative standpoint,, what ,does such criti- 
cism mean? Feminist researchers, 'Westkott (1979) and 
Eichler (1980) , pointed our that in traditional studies 
criticism focused on" the questions asked ; the techni- 
ques used; and the Conclusions draym. In this project 
although not traditional 9 reactions were indeed of this 
nature. Nevertheless ,* when sufficient criticisms were 
made, prior to the completion of the questionnaire, the 
final instrument was much improved and- the attitude of 
participants, while completin^xthe form, was very posi- 
tive.- 4 - Thus, it could be concluded^Jthat a stance of in- 
dependence and. objectivity by an eval^tor in a femi- 
nist setting would haye__been detrimental . to^tA^assess- 
ment process and would not have met the Special evatiXa 
tive needs of institute personnel and participants^ 

TheNf indings of this study also indicated that il- 
luminative evaluation showed promise as one" eva luative 
methodology compatible with feminist! activity and theo- 
retical concerns- In line with the purposes of evalu- 
ation which- the director of. the Institute had in mind 
for year one, the illuminative evaluation mociel enabled 



a report to be developed which would serve to offer 
both qualitative and quantitative data useful to the 
designing of year two 'of the Institute; provide in- 
formation 'for continued funding of the project; and 
include documentation of the first : year of the Insti- 
tute. 
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